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Abstract. Given a right eigenvector x and a left eigenvector y associated with the same eigen- 
value of a matrix A, there is a Hermitian positive definite matrix H for which y = Hx. The matrix H 
defines an inner product and consequently also a field of values. The new generalized field of values 
is always the convex hull of the eigenvalues of A. Moreover, it is equal to the standard field of values 
when A is normal and is a particular case of the field of values associated with non-standard inner 
products proposed by Givens. As a consequence, in the same way as with Hermitian matrices, the 
eigenvalues of non-Hermitian matrices with real spectrum can be characterized in terms of extrema 
of a corresponding generalized Rayleigh Quotient. 
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1. Introduction. We propose a new generalized field of values which brings all 
the properties of the standard field of values to non-Hermitian matrices. For a matrix 
A G C" xn the (classical) field of values or numerical range of A is the set of complex 
numbers 

F(A) = {x*Ax : x G C™, x*x = 1}. (1.1) 

Alternatively, the field of values of a matrix A £ C TIX " can be described as the region 
in the complex plane defined by the range of the Rayleigh Quotient 

p(x,A) = ^, Vx^O. (1.2) 

x*x 

For Hermitian matrices, field of values and Rayleigh Quotient exhibit very agreeable 
properties: F(A) is a subset of the real line and the extrema of F(A) coincide with 
the extrema of the spectrum, cr(A), of A. Equivalently, the vector v maximizing 
p(x, A) is an eigenvector associated with the largest eigenvalue, p(v,A), of the matrix 
A. Therefore, every eigenpair of A is the solution of a maximization-minimization 
problem in some constraint subspace. Such results are Rayleigh-Ritz and Courant- 
Fischer's theorems [H §4.2]. 

Unfortunately, as the next example shows, for non-Hermitian matrices such pleas- 
ant properties no longer hold. Even if all the eigenvalues of A would be real. 

Example 1.1. Figure \1.1\ and depict the classical field of values of a Her- 
mitian matrix, A, and a non-Hermitian matrix, B, with real eigenvalues respectively. 
Although the eigenvalues of the matrix B lay on the real line such as those of A, the 
extrema of the field of values and of the spectrum no longer coincide. 



Within the last sixty years several generalizations for the field of values were 
proposed (see [5J §1.8] for a more complete list). Each of the generalizations attempted 
to replicate one or more of the properties of the classical field of values. In 1952, 
Givens [2] proposed the field of values of A G C™ xn associated with a generalized 
inner product. For any A G C nx ™ Givens field of values is the set 

F H {A) = {x*HAx : x G C" x ", x*Hx = 1, H is Hermitian positive definite}. (1.3) 
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Givens intent was to extend F(A)'s invariance under unitary similarity transfor- 
mations to arbitrary similarity transformations. If, for arbitrary nonsingular V, 
B = VAV^ 1 is a similarity transformation of A, then F H (A) = F(B) for H = V*V. 
The converse is also true. Givens proves two important statements: 



Theorem 1.2. The intersection of the regions Fh(A) for all positive definite 
matrices H is the minimum convex polygon, Co{a{A)) containing all the roots of A. 

Theorem 1.3. Fh(A) = Co{a{A)) for some positive definite hermitian matrix 
H if and only if the elementary divisors corresponding to roots lying on the boundary 
of Co{o~{A)) are simple. 

When A is normal Fjj(A) — F(A) — Co(a(A)), the convex hull of the spectrum 
of A. It is unclear, however, if for arbitrary A, Givens knew which H (if any) satisfies 
F H (A) = Co(a(A)). 

A few years later, Bauer [I] proposed a new generalization. Bauer extended the 
original formulation of the field of values to norms which need not be associated with 
inner products 

F H (A) ee {y*Ax : x,y G C" and \\y\\ D - = y*x = 1} (1.4) 

where \y\ D stands for the dual (vector) norm (see [5]). Bauer's generalized field of 
values makes use of different (though related) vectors at the right and left sides of A. 
Those vectors are dual pairs with respect to the vector norm || • |j. Moreover, 
depends only on the norm and A and not on an inner product. However, and although 
Bauer's generalized field of values always contains <j{A), it is not always convex [81IT2]. 
Moreover, according to the authors in j5] the field of values from Givens is a special 
case of Bauer's field of values. 

A more recent generalization is the q-field of values [7] 

F q (A) ee {y*Ax : x, y e C\y*y = x*x = 1, and y*x = q}. (1.5) 

Also for F q (A) different vectors x and y are used for the inner product {Ax, y). They 
must, however, satisfy an extra constraint y*x — q. The convexity property holds for 
q G [0, 1] and A G C" x ™ with n > 2 |5 ]. Finally, if q = 1, the g-field of values reduces 
to the standard field of values. 
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1.1. Notation and Outline. Although most of the notation we use is consid- 
ered standard, we believe to be useful to clarify some assumptions. We represent 
eigenvalues by the Greek letters A and choose to label them in non-decreasing order 
of magnitude: minA = Ai < A2 < ... < A n _i < A„ = max A. The spectrum of a 
matrix A, the set of the eigenvalues of A, is denoted by cr(A). Letters at the end of 
the alphabet represent vectors and these will always be column vectors. Lower case 
Latin letters i and j denote indices. The conjugate transpose of a matrix X is denoted 
by X* , also for X real. With I n we mean the n x n identity matrix and by ej its 
jth column. To minimize clutter we use A~*, if necessary, to mean (A*) -1 = (A -1 )* 
while (x,y) will denote the standard inner product between vectors i,i/e C". 

In [J5]we define the new generalized field of values giving emphasis to the relations 
between left and right eigenvectors of non-Hcrmitian matrices. In [J3]we prove impor- 
tant properties of the generalized two-sided field of values and show how, for particular 
types of matrices, it naturally reduces to the classical field of values and to some of 
the early generalized approaches. Section|4]describes the two-sided Rayleigh Quotient 
illustrating some of its properties and relating it with the newly defined generalized 
two-sided field of values. Here we show how the properties of the classic Rayleigh 
Quotient carryover to the generalized field of values once the correct inner product 
is chosen. Finally, in that same section, we show how using the new definitions the 
extremal properties of the standard Rayleigh Quotient known for Hcrmitian matrices 
can be extended to the non-Hcrmitian case for matrices with real eigenvalues. 

2. A new generalized field of values. If A is a Hermitian matrix and Agl 
an eigenvalue of A, the set of vectors v satisfying Av = Xv is equal to the set of vectors 
w satisfying w*A — Xw* . These are the right and left eigenvectors associated with the 
eigenvalue A. If A is non-Hermitian, however, that is not necessarily the case. None 
of the generalizations of the field of values discussed in section ^T] focuses on capturing 
the dynamics of left and right eigenvectors associated with the same eigenvalue of a 
non-Hcrmitian matrix. Assume A is nondefective. Then there exists a nonsingular 
matrix V whose columns are right eigenvectors of A and that satisfies 

V' X A = AV^ 1 and AV = VA. 

The rows of V~ x are the left eigenvectors of A while A is the diagonal matrix of the 
eigenvalues of A. A crucial observation is the following 

Lemma 2.1. Let v r ,vi G C nx ™ & e the right and left eigenvectors corresponding 
to the same eigenvalue X{ of a nondefective matrix A G C" xn . Let A = VAV^ 1 be 
a diagonalization of A and denote by A the diagonal matrix of the eigenvalues. Take 

V to be a matrix whose columns are eigenvectors of A scaled appropriately. Then v r 
and vi satisfy 

V*vi = V~ 1 v r = Ci or equivalently v r — VV*vi — Va. 

Proof. The nondefectiveness of A guarantees the existence of a nonsingular matrix 

V for which A — VAV" 1 where A = diag(X%, . . . , A„). As a consequence, the matrix 
A is complex symmetric. Therefore, for v r and vi as above, (v*V)* = V~ 1 v r = ei or 
equivalently, vi = (VV*)~ 1 v r = V~*ei or yet v r = VV*vi = Vet. □ 

The Hcrmitian case (A* = A), emerges as a particular occurrence of the previous 
lemma. The matrix V is then unitary, i.e. V* — V~ x and as a consequence the 
statement shortens to v r — vi = Vej. 
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Similar to the case of Bauer's and of the g-field of values, so does our generalized 
two-sided held of values uses different vectors on the right and left of A. Those vectors 
y and x now form a dual pair with respect to an eigenvalue of A. The definition that 
follows introduces the generalized two-sided held of values. 

Definition 2.2 (Generalized two-sided field of values). For a nondefective ma- 
trix A = VAV , the generalized two-sided field of values is the set of complex numbers 

G(A) ee {y*Ax :x,yeC n ,y = (VV*)~ 1 x and y*x = 1}. (2.1) 



3. Properties of the generalized two-sided field of values. We will assume 
that A can be diagonalized as A — VAV^ 1 and that G(A) is defined as in (|2.1[) . The 
first two properties to come are standard properties for fields of values (see also [B]). 
Further on we introduce some particular characteristics of G(A). 

Property 3.1. For any complex number a, 

G(A + al) = G(A) + a. 

Proof. That the basis of eigenvectors of A and A + al is the same follows from 
V- 1 {A + aI)V = A + al = D. Therefore, 

G(A + al) = {y*(A + al)x : y = (VV*)~ 1 x, y*x = 1} 
= {y*Ax + ay*x : y = (VV*)~ 1 x, y*x = 1} 
= {y*Ax : y = {VV*y 1 x, y*x = l} + a = G(A) + a. □ 

Property 3.2. For any nonzero complex number a, 

G(aA) = aG(A). 

Proof. 

G(aA) = {y*(aA)x : y = (VV*)- 1 x,y*x = 1} 
= {ay* Ax : y = (VV*)~ 1 x, y*x = 1} 
= a{y*Ax : y = {VV'^x, y*x = 1} = aG{A). □ 

Property 3.3. For all nondefective matrices ieC" x ", 

G(A) = F H (A) 

with H = {VV*)- 1 . Proof. Define H = (VV*)- 1 . The matrix H' 1 = VV* is 
positive definite and therefore so is H . Consequently, 

y*Ax = x*(VV*)~ 1 Ax = x*HAx and l = y*x = x*Hx. □ 

As such, G(A) is a particular case of Fjj(A). However, as the next properties 
show it possesses very agreeable characteristics. 

Property 3.4. For all nondefective A e C nxn of the form A = VAV^ 1 , 

G(A) = F(A). 
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Proof. Let z — V 1 x. Then, 
y*Ax = xV~*V~ l Ax = z*V~ 1 AVz = z*Az and 1 = y*x = z*z. □ 

Property 3.5. Let H(A) = \ {A + A*), 

G{H(A)) = &F(A). 

Proof. Because H(A) is normal it follows from Property 13.81 that 

G{H(A)) = F{H(A)) = HtF(A), 

where the last equality follows from the properties of the classical field of values. □ 
Property 13.51 contrasts with the equivalent one for the classical field of values. Unlike 
the latter, the projection of G(H(A)) onto the real axis for non-Hermitian matrices 
is not orthogonal but oblique and so G(H(A)) ^ $IG(A). 

It follows from Propertv l3.4l and from the properties of the classical field of values 

that 

Property 3.6. G(A) is a compact and convex set. 

Not only is the set compact and convex as it also possesses a very important 
property related to the spectrum of A. 

Property 3.7. For all nondefective A e c nx ™ 

G(A) = Co{a{A)). 

where Co{a{A)) denotes the convex hull of the spectrum of A. 

Proof. If A is nondefective, there exists a nonsingular matrix V for which A = 
V^AV, where A = diag{\\, . . . , A n ) is the diagonal matrix of eigenvalues of A. 
By definition G(A) = F(A) where the field of values of A is the set of all convex 
combinations of the diagonal elements of A: 

n n 

z*Az = ^2\zi\ 2 Xi where ^|z. i | 2 = l. 

i— 1 i 

The proof follows from the definition of convex hull of a set S as the set of all convex 
combinations of finitely many points of S. □ 
Property 3.8. For all normal A e C nxn , 

G(A) = F(A). 

Proof. As a consequence of the normality of A the matrix V is unitary, i.e. 
V' 1 = V*. Therefore, 

G{A) = F(V- 1 AV) = F(V*AV) = F(A) 

where the last equality results from the unitary invariance property of the classical 
field of values (|5J Property 1.2.8]). □ 

In summary, G(A) is, for any nondefective matrix, always convex and always the 
convex hull of the eigenvalues of A. This gives the following corollary to Theorem 1 1.2 1 
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Corollary 3.9. When V is the matrix of eigenvectors of A and H = (VV*) 1 , 
the region Fg(A) is the intersection of the regions F^(A) over all positive definite H. 

The matrix H = (VV*) -1 , however, is not the only matrix H for which Fjj = 
Co(a(A)) as the next example shows. 

Example 3.10. Let A be a nondefective matrix such that both A = VAV -1 and 
V can be partitioned as follows 



A : 



Ai 



A 2 



V 



I 



v 2 



where Ai is diagonal and F(A%) C -F(Ai). Then for both H\ = I and H% = (VV*) 1 
,F Hl =F H2 =Co(o-(A)). 

Note: So far we have assumed that the matrix A must be nondefective. However, 
as Theorem II .31 from Givens shows, this requirement can be made less tight. In fact, 
let A be any (square) matrix whose elementary divisors corresponding to eigenvalues 
lying on the boundary of Co(a(A)) are simple. Let J be the bidiagonal matrix of the 
Jordan normal form of A and partition it as follows 



J = 



A 



T 



Here, A is the (diagonal) matrix of eigenvalues of A located on the boundary of 
Co(a(A)) and T the bidiagonal matrix containing the remaining roots. Partition W 
in a similar fashion as W = \W\ W2] where Wi contains the set of linear indepen- 
dent eigenvectors associated with the eigenvalues in A and W2 the set of generalized 
eigenvectors associated with the eigenvalues on the diagonal of T. Then, 

G(A) = {y*Ax :x,yE C n ,y = (WW*)~ 1 x, y*x = 1}. 

To end this section we prove a result on the location of G(A) in the complex 
plane when A is positive definite. First, however, we need to extend the definition of 
positive definite matrices. It is widely accepted that a positive definite matrix is a 
Hermitian matrix, A £ C" xn , for which 



"Ax > 0, 



for all ^ x £ C" 



The previous definition neglects non-Hcrmitian matrices with (real) positive eigen- 
values. Furthermore, there is no agreement on the literature on what the proper 
extension to the non-Hermitian case should be. We wish to contribute to the discus- 
sion by proposing a definition consistent with the earlier generalization of the field of 
values. In this way, 

Definition 3.11 (Positive definite matrix). A nondefective matrix A G C" xn 
diagonalizable as A = VAV -1 is said to be positive semidefinite if 

y*Ax > 0, for all ^ x E C n and (VV*)y = x, (3.1) 

and positive definite if in addition, 

y*Ax > 0, for all ^ x e C" and (VV*)y = x. (3.2) 

This definition is consistent with the generalization of the inner product. This is due 
to the fact that Equations (|3.1[) and (|3.2[) are equivalent to 



(Ax,x) H > 



and 



(Ax,x) H > 



(3.3) 
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respectively, in the inner product denned by the matrix H = (VV*) -1 , the i/-inner 
product. To satisfy the alternative characterization that ^4's eigenvalues are real and 
positive, we must have H — (VV*)^ 1 . For normal matrices H — I, thus reverting to 
the standard definition. We now enunciate the property 

Property 3.12. If A is positive definite then G(A) C {z : Re z > 0}. 

Proof. If A is positive definite in the sense of the definition above then A is either 
Hermitian positive definite or has real positive eigenvalues. That the property is true 
for the Hermitian case follows from Property 13.81 For non-Hermitian matrices the 
result is given by Property 13.71 □ 

Example 3.13. Figures \3J\ and HOI show the standard, F(A), and the Gener- 
alized two-sided field of values of a non-Hermitian matrix A with real eigenvalues. 
Fiaures \3.3\ and \3.4\ represent a similar situation for a random matrix B £ R nx ™ with 
some complex eigenvalues. 

4. Generalized two-sided Rayleigh Quotient. Such as the standard field of 
values is related to the standard Rayleigh Quotient so the generalized field of values 
is connected with a generalized Rayleigh Quotient. Recall that given a Hermitian 
matrix A, and an n-dimensional vector x the standard Rayleigh Quotient, p(x, A) is 
the function 

x* Ax 

p{x,A) = —^, x^O. 
x*x 
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For the nonsymmetric case, the situation is more delicate. Instead of the quadratic 
form, the loss of symmetry requires us to handle a bilinear one. An intuitive general- 
ization would be 

11* At 

p(y,x,A) = V-^, y*x^0 (4.1) 

which is consistent with the ones given in [§J In particular part III] and follow-ups 
such as PHI |3]. A consequence of the loss of symmetry, however, is that with the 
generalization just defined and even if all the eigenvalues of A would be real, p{y, x, A) 
is no longer maximized at the largest eigenpair of A (or, for what is worth, minimized 
at the smallest). The correct generalization requires an extra constraint. 

Definition 4.1 (Generalized two-sided Rayleigh Quotient). Assume the ma- 
trices A, M G C" x ™ are nondefective and let A be diagonalizable as A — VAV . 
Denote by y and x two n-dimensional vectors in C such that y*x ^ 0. The general- 
ized two-sided Rayleigh Quotient of x and y is then defined as the function 

y* Ax 

p(y,x,A) = , with y*Mx^0 and VV*y = x. 

y* Mx 

The matrix M in the denominator is a Hermitian and positive definite matrix in the 
ii-inner product, where H = (VV*) -1 . In addition, the standard Rayleigh Quotient 
is obtained for particular choices of x,y and M, namely with VV* = M — I. For 
what follows, we assume, that M = I. 

4.1. Properties of the Generalized two-sided Rayleigh Quotient. It is 

a known fact that the standard Rayleigh Quotient, p(v r ,A) of a normalized right 
eigenvector, v ri associated with eigenvalue of a matrix A satisfies p(v r ,A) = Xi. 
An equivalent statement is true for the left eigenvector V[ associated with A,;, that 
is, p(vi,A) = Xi. Equivalently, for the generalized two-sided Rayleigh Quotient, 
p(vi,v r ,A) 

Lemma 4.2. Let v r ,vi G C" be the normalized right and left eigenvectors asso- 
ciated with the same eigenvalue Xi of a matrix A S C nxn . Define p(vi,v r , A) as the 
generalized two-sided Rayleigh Quotient of vi and v r . Then, 

V?Av r V?Avi V*Av r . 

p(vi,v r ,A) = — = -4— = — - — = Xi. 

Vjv r V^Vl V*V r 

Proof. Notice that because v r and vi are the right and left eigenvectors corre- 
sponding to Xi the constraint vi = (VV*)~ 1 v r is superfluous. Moreover, v*v r ^ 0. 
Assume = ||iv| = 1, then the last two equalities follow from the definition of 
right and left eigenvector. For if v* A = A^ z * and Av r = XiV r then 

v* Avi = Xi\\vi\\l and v*Av r = Xi\\v r \\l. 

As for the second equality, 

p{vi,v r ,A) = -4 — = — f — =Xi. D 

v l v r V^Vr 

Given a nonzero vector x, the standard Rayleigh Quotient is the scalar p(x, x, A) 
for which x _L (A — p(x, x, A)I)x. Likewise, the generalized two-sided Rayleigh Quo- 
tient is the scalar p(y, x, A) for which 



y ± (A- p(y,x,A)I)x and x ± (A* — p(y, x, A)I)y. 
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The vector pjv, x, A)x is the oblique projection onto x and orthogonally to y of Ax 
while p(y, x, A)y is the oblique projection onto y and orthogonally to x of A*y. More- 
over, recall that given a vector x 6 C" with norm one and a scalar /i£C, the measure 
of how close (p,x) is of being an eigenpair of A is ||r||| = (r, r) where r = Ax — px. 
In truth, we need only x, as the scalar p minimizing ||r||2 is no other than p(x,x, A) 
(see [10] or [H]). The non-Hermitian case is slightly more involved since the left and 
right eigenvectors differ. Therefore, a triplet (/i, x, y) consisting of a scalar and two 
n-vectors is needed to determine r y = y* A — py* and r x — Ax — px. 

Standard properties of the classic Rayleigh Quotient such as homogeneity and 
translation invariance are carried over, in a straightforward way, to the generalized 
two-sided Rayleigh Quotient: 

• Homogeneity: p{av 1 j3w, A) = p(v,w,A) and 

v*/3Aw v*Aw ., 
p(v, w, f3A) = = 13—— = Pp(v, w, A); 

v*Bw v* Bw 

• Translation Invariance: p(v, w, A — pi) = p(v, w, A) — p. 

In addition, the boundedness property that fails when taking p(y,x,A) (see |10j ) 
is satisfied for the generalized two-sided version we propose (cf. Property 13.61 and 
13. 7p . As is an equivalent to the minimal residue property of the standard Rayleigh 
Quotient. This also in contrast to the approach given by Equation (|4.ip without the 
extra constraint. In this situation, we can only guarantee that for a nonsymmetric 
matrix A with real eigenvalues, the quantity p(y, x, A) minimizes the inner product 

• Minimal Inner product : 

For A G M™ xn with real eigenvalues and given R™ such that y*x ^ 0, 

for any scalar p 

{A*y - py*)*{Ax - px) > y*A 2 x - p 2 y*x, 

with equality only when p = p = p{y, x, A). 
Proof. Set p = p(y, x, A). 

(A*y — py*)*(Ax — px) = y*AAx — py* Ax — py*Ax — p 2 y*x 
= V*x (y*A 2 x/y*x + (p - p) 2 - p 2 ) 
>y A x - p y x, 

with equality only when p = p. □ 

If, however, the extra constraint (y — (VV*)~ 1 x) is used 
Lemma 4.3 (Minimal residue norm). Let A — VAV' 1 and H 

u, v =^ and for any scalar p 

\\Au-pu\\ 2 H >\\Au\\ 2 H -\\pu\\ 2 H and \\v* A- pv*\\% >\\v* A\\% - \\pv*\\% 

with equality only when p = p = p{y,x,A). 

Proof. Set p — p(y, x, A), H = (VV*) -1 and r u — Au — pu and recall that H is 
a Hermitian positive definite matrix. 

|| r„ || |f = (HAu - pHu)*(Au - pu) 

= u*A*HAu - pu*HAu - pu*A*Hu + ppu*Hu 

= (u*Hu) [(u*A*HAu)/(u*Hu) + (p - p){p - p) - pp] 

>\\Auf H -\\puf H 



= (VV*)- 1 . Given 
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where \\z\\ 2 H = (z,z)h = (z,Hz) for any z G C™. We replace r u in the proof by 
r v = v* A — pv* to obtain the second part of the statement. □ 

Because the range of p(y, x, A) is a subset of the range of p(y, x, A) we cite a 
result from B.N. Parlett |10j for the stationarity property in the form of a lemma. 

Lemma 4.4. The generalized two-sided Rayleigh Quotient, p(y,x,A), is station- 
ary if and only if y and x are the left and right eigenvectors of A associated with 
eigenvalue p(y, x, A) and y*x ^ 0. 

Proof (see pH §11]). 

4.2. Extrema of the generalized two sided Rayleigh Quotient. Similar 
to the normal case, the extrema of the generalized two-sided Rayleigh quotient are 
the largest and the smallest eigenvalues of those non-Hcrmitian matrices whose eigen- 
values are real. This is the topic of the next theorem whose Hermitian version is 
attributed to Rayleigh and Ritz. 

Theorem 4.5. Let A e <C nxn be a non- Hermitian, nondefective matrix. Let 
A = V~ 1 AV be the diagonal matrix of eigenvalues of A and assumed to be real and 
ordered as X\ < . . . < A„ . Then, 

Xiy*x < y*Ax < \ n y*x for all x e C" and y = {VV*)~ l x (4.2) 

Amax = A„ = max - f X = max y* Ax (4.3) 

y'x^O y*X y*x=l 
y=(VV*)- 1 x " y = {VV)- 1 x 

X m in = \\ = mm - f X = min y*Ax. (4.4) 

y""x=£0 y*X y*x = l 

y=(VV*)- 1 x " y* = (VV)- 1 x 



Proof. From Lemma [2.1l we know that if vi and v r are the left and right eigenvector 
of A corresponding to the eigenvalue A^, then (v^V)* — V v r — or equivalently, 
Vi = {VV*)~ l v r . Now, for any x £ C™ and y = (VV*)~ 1 x 



y*Ax = y^VAV^x = x* (V" 1 )* AV^x = ^ \i\iV~ 1 



x)i\ 2 . 



Because each term \ (V 1 x)i\ 2 is nonnegative, this is a convex combination of the real 
numbers Ai for i = 1, . . . , n. Therefore 

n n n 

Xmin E \( V ~ lx )*\ 2 < V* Ax = E X i\( V ~ lx )i\ 2 < X max £ | ( V" ~ 1 a;) ^ | 2 . 



For z — V 1 x notice that \(V 1 x)i\ 2 — \\z\\ 2 — z*z — y*x. Consequently, 

i=i 

Xiy*x < y*Ax < X n y*x. 

Equality occurs when y and x are the right and left eigenvalues of A corresponding 
to Ai or A„ as appropriate (see Lemma |4.2[) . In other words, 

y*Ax y*Ax 
mm = Ai and max = A„. 

y*x=£0 y*X y'x^O y* X 

y=(VV)- 1 x " y=(VV)- 1 x 
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In addition y and x can be normalized so that y*x — 1 resulting in 

max y* Ax = A„ and min y* Ax = \\. □ 

y*x— 1 y*x— 1 

y=(VV*)- 1 x y=(VV)- 1 x 

Two additional notes are now called for. One to say that for z = V~ 1 x and 
A = V- X AV 

max y* Ax = max z* Az 

yx=i || z ||=i 

y— (W*) 1 a; 

Therefore, by symmetry of A, what was just developed for x can in a similar manner 
be done for y by setting x — VV*y. The second to draw attention to the fact that 

y* Ax y* Ax . y* Ax . y* Ax 
max < max and mm > mm 

y*x=£0 y*X y*x=£0 y* X y*x^0 y*X y*x^0 y* X 

y=(VV)- 1 x " y={VV*)- 1 x 

as a result of the extra constraint on the left-hand side. 

The extra restriction to the set over which the maximum is taken, can be seen 
as the nonnormal equivalent of the condition y — x, since in the normal case, V is 
orthogonal and VV* = I. Unfortunately the price to pay for nonnormality is high, 
rendering limited practical use to the previous results. The matrix V is, in general, 
not known and if otherwise there would no longer be the need for determining right 
and left eigenvectors. In theoretical terms, however, it allows for the generalization 
of the variational characterization of the eigenvalues to non-Hermitian matrices with 
real eigenvalues. We are now able to generalize Courant-Fischer minimax theorem to 
real non-Hermitian matrices with real eigenvalues. 

Theorem 4.6. Let j and n be integers such that 1 < j < n and let A e C nxn 
be a nondefective matrix. Assume the eigenvalues of A to be real and ordered as 
Ai < . . . < A„. Let S 3 denote a j-dimensional subspace o/C™. Then, 

\ ■ y* Ax ■ p* A i (a k\ 

A, = mm max = max mm (4-5) 

* xesi y*x sn-j+i qe s n -i+ x p*q 

y=(VV)- 1 x p=(W*)-lg 

Proof. We have shown earlier that with the variable transformation z = V~ x x 

, as y* Ax z*Az . , . 

p{y,x,A)= max — - — = max — - — = p(z, z, A). 

y*x=iO y*X z^O Z*Z 

y=(VV*)- 1 x 

By the basis theorem and because the columns of V form a basis for C™, though not 

— j ~ j _|_ J_ ^ j ~ J — |— X 

orthogonal, S n S ^ 0. There exist, thus, a vector z e S n S for which 

min p(v,v,\) < p(w,w,A) < max /)(«, u, A). 
ves"- 3+1 ' ues J 

Because the inequalities are valid for all choices of S"' and S J ' + we have 
max min p(v, v, A) < min maxp(w, u, A). 
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In the basis of the original matrix this means 

max min p(p, q, A) < min max p(y,x,A). 

p =(vv)- 1 q j=(yv)-'i 

Equality follows from taking S 3 = V 1 the span of the first j (right) eigenvectors and 
gn.-i+i ^- ne S p an f the i as t n — j + I (right) eigenvectors, rendering 

A, = max min p(p,q,A) and A, = min max p(y,x,A). □ 

P =(yy*)-i 9 s=(w) _1 i 

Acknowledgments. The author wishes to thank Jan Brandts for his valuable 
comments on an early version of this paper. 
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